Investigate a Dataset - European Soccer Database - Spain LIGA BBVA¶
Data Set¶
European Soccer Database
Overview and Notes¶
This soccer database comes from Kaggle and is well suited for data analysis and machine learning.
It contains data for soccer matches, players, and teams from several European countries from 2008 to 2016.
This dataset is quite extensive, and we encourage you to read more about it here.
The database is stored in a SQLite database. You can access database files using software like DB Browser.
This dataset will help you get good practice with your SQL joins.
Make sure to look at how the different tables relate to each other.
Some column titles should be self-explanatory, and others you’ll have to look up on Kaggle.
Sources¶
http://pandas.pydata.org/pandas-docs/version/0.9.1/visualization.html
https://matplotlib.org/gallery.html
https://www.udacity.com/course/intro-to-programming-nanodegree--nd000
https://www.kaggle.com/hugomathien/soccer
Example Questions¶
How many goals has each team scored home, away and total on each season?
How many goals has each team allowed home, away and total on each season?
How teams improved or deteriorated over time and what teams improved or deteriorated the most and the least in terms of goals scored or allowed over time?
How many goals has each team allowed home, away and total, from other teams, over all seasons?
Is there such a thing a home advantage?
Which players had the most penalties?
What team attributes lead to the most victories?
Import data analysis modules¶
In [1]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import sqlite3
import seaborn as sns
In [2]:
connection = sqlite3.connect('database.sqlite')
In [3]:
query = '''
select
Country.name as Country,
League.name as League,
Match.season as Season,
date(Match.date) as Date,
Team_home.team_long_name as Team_Home,
Team_away.team_long_name as Team_Away,
Match.home_team_goal as Team_Home_Goal,
Match.away_team_goal as Team_Away_Goal
from
Country, League, Team as Team_home, Team as Team_away, Match
where
Country.id = League.country_id AND
Country.id = Match.country_id AND
League.id = Match.league_id AND
Team_home.team_api_id = Match.home_team_api_id AND
Team_away.team_api_id = Match.away_team_api_id AND
Country = 'Spain' AND
League.name = 'Spain LIGA BBVA'
order by
Match.date
-- limit 10;
'''
#Run query and store results in a dataframe
spain_liga_bbva_df = pd.read_sql(query,connection)
Print dataframe columns¶
In [4]:
print(spain_liga_bbva_df.columns)
Create helpful functions that will be used repeatedly for resetting an index of an input series, store result in a dataframe, pivot it so that each Season is a separate column and finally plot its input columns over its input index¶
In [5]:
"""
reset_index_pivot_stacked_bar_plot_series(Args) is used for resetting an index of an input series, store result in a dataframe, pivot it so that each Season is a separate column and finally plot its input columns over its input index.
Args:
plot_type: Type of chart plotted, 'stacked_bar' or 'heat_map'
season_team_scored_allowed_sum: input pandas Series containing Season, Team and sum of goals scored or allowed for each season
index: 'Team_Home' or 'Team_Away' or 'Team' index used for x label plotting
columns: Columns to pivot resulting dataframe, usually 'Season', used as x label subplotting
values: 'Team_Home_Goal' or 'Team_Away_Goal' or 'Team_Goal' or 'Team_Allowed' columns used as values of a plot
title: Title of a plot
xlabel: X axis label of a plot
ylabel: Y axis label of a plot
Returns:
Pivoted DataFrame with each Season in a separate column.
Raises:
None
"""
def reset_index_pivot_plot_series(plot_type, plot_color, season_team_scored_allowed_sum, \
input_index, input_columns, input_values, title, xlabel, ylabel, reset_flag, pivot_flag):
if reset_flag:
#Reset index of series so 'Season' column is fully populated, store in pandas DataFrame
season_team_scored_allowed_sum = season_team_scored_allowed_sum.reset_index()
if pivot_flag:
#Pivot dataframe so that each Season is a separate column, needed for multi-series plots
season_team_scored_allowed_sum = season_team_scored_allowed_sum.pivot(index=input_index, columns=input_columns, values=input_values)
if plot_type == 'stacked_bar':
#Plot a stacked bar of final DataFrame with details provided
season_team_scored_allowed_sum.plot(kind='bar', stacked=True, figsize=(20, 15))
elif plot_type == 'heat_map':
#Plot a heat map of final DataFrame with details provided
plt.subplots(figsize=(20,15))
sns.heatmap(season_team_scored_allowed_sum, annot=True, cmap=plot_color)
else:
#Plot a stacked bar of final DataFrame with details provided
season_team_scored_allowed_sum.plot(kind='bar', stacked=True, figsize=(20, 15))
plt.title(title)
plt.xlabel(xlabel)
plt.ylabel(ylabel)
plt.show()
#Return final DataFrame
return season_team_scored_allowed_sum
Compute home and away goals scored per season and add them up, store in pandas Series¶
In [6]:
season_team_home_goal_sum = spain_liga_bbva_df.groupby(['Season', 'Team_Home'])['Team_Home_Goal'].sum()
season_team_away_goal_sum = spain_liga_bbva_df.groupby(['Season', 'Team_Away'])['Team_Away_Goal'].sum()
season_team_goal_sum = season_team_home_goal_sum + season_team_away_goal_sum
print type(season_team_home_goal_sum)
print season_team_home_goal_sum.head(30)
Interest 1a: We are interested in computing and plotting the goals scored home on each season for each team. This way we can see if a team is performing very well in terms of goals scored home. We would also like to see if any team has performed better or worse over time. A simple bar chart with each team on the x-axis and goals scored home on the y-axis, aggregated over each season should help us get an idea.¶
Question 1a: How many goals has each team scored home on each season?¶
Answer 1a: Home goals scored per Season¶
In [7]:
season_team_home_goal_sum = reset_index_pivot_plot_series('stacked_bar', '', season_team_home_goal_sum, \
'Team_Home', 'Season', 'Team_Home_Goal', 'Home goals scored per Season', 'Team', 'Home Goals Scored', True, True)
season_team_home_goal_sum.head()
Out[7]:
Conclusion 1a: This graph shows two best performing team in terms of goals scored home, FC Barcelona and Real Madrid. Looking at the chart though is not that easy to notice any improvement or deterioration over time for each team. This is a limitation of this specific chart that we will explore next.¶
If we display the same information, but this time using a heat map, we can clearly see how each team has performed over time. For example we can see that both FC Barcelona and Real Madrid peaked their performance in the 2011/2012 season and are getting ready for a comeback in the 2015/2016 season.¶
In [8]:
season_team_home_goal_sum = reset_index_pivot_plot_series('heat_map', 'YlGnBu', season_team_home_goal_sum, \
'Team_Home', 'Season', 'Team_Home_Goal', 'Home goals scored per Season', 'Season', 'Team', False, False)
Interest 1b: Next, we are interested in computing and plotting which team performed better or worse over time in terms of goals scored home. A heatmap chart with each team on the y-axis and goals scored home differentials (goal difference from season to season) as values, with each season on the x-axis, should give us an idea.¶
Question 1b: How teams improved over time and what teams improved the most and the least in terms of goals scored home over time?¶
Answer 1b: Home goals scored over Seasons¶
In [9]:
season_team_home_goal_sum_diff = season_team_home_goal_sum.diff(periods=1, axis='columns')
season_team_home_goal_sum_diff = reset_index_pivot_plot_series('heat_map', 'cool', season_team_home_goal_sum_diff, \
'Team_Home', 'Season', 'Team_Home_Goal', 'Home goals scored differentials over Seasons', 'Season', 'Team', False, False)
Conclusion 1b: This graph shows two best performing team in terms of goals scored home differentials, FC Barcelona and Athletic Club de Bilbao. Barcelona had a stellar improvement in the 2011/2012 season compared to its previous season. Teams like Mallorca and Valencia on the other hand had a bad season in terms of our metric in 2010/2011 and 2015/2016 seasons correspondingly.¶
Interest 2a: Similarly to goals scored home, we are also interested in computing and plotting the goals scored away on each season for each team. This way we can see if a team is performing very well in terms of goals scored away. We would also like to see if any team has performed better or worse over time. A simple bar chart with each team on the x-axis and goals scored away on the y-axis, aggregated over each season should help us get an idea.¶
Question 2a: How many goals has each team scored away on each season?¶
Answer 2a: Away goals scored per Season¶
In [10]:
season_team_away_goal_sum = reset_index_pivot_plot_series('stacked_bar', '', season_team_away_goal_sum, \
'Team_Away', 'Season', 'Team_Away_Goal', 'Away goals scored per Season', 'Team', 'Away Goals Scored', True, True)
season_team_away_goal_sum.head()
Out[10]:
Conclusion 2a: This graph shows two best performing team in terms of goals scored home, FC Barcelona and Real Madrid. Looking at the chart though is not that easy to notice any improvement or deterioration over time for each team. This is a limitation of this specific chart that we will explore next.¶
If we display the same information, but this time using a heat map, we can clearly see how each team has performed over time. For example we can see that both FC Barcelona and Real Madrid peaked their performance in the 2012/2013 and 2014/2015 season correspondingly.¶
In [11]:
season_team_away_goal_sum = reset_index_pivot_plot_series('heat_map', 'YlGnBu', season_team_away_goal_sum, \
'Team_Away', 'Season', 'Team_Away_Goal', 'Away goals scored per Season', 'Season', 'Team', False, False)
Interest 2b: Next, we are interested in computing and plotting which team performed better or worse over time in terms of goals scored away. A heatmap chart with each team on the y-axis and goals scored away differentials (goal difference from season to season) as values, with each season on the x-axis, should give us an idea.¶
Question 2b: How teams improved over time and what teams improved the most and the least in terms of goals scored away over time?¶
Answer 2b: Away goals scored over Seasons¶
In [12]:
season_team_away_goal_sum_diff = season_team_away_goal_sum.diff(periods=1, axis='columns')
season_team_away_goal_sum_diff = reset_index_pivot_plot_series('heat_map', 'cool', season_team_away_goal_sum_diff, \
'Team_Away', 'Season', 'Team_Away_Goal', 'Away goals scored differentials over Seasons', 'Season', 'Team', False, False)
Conclusion 2b: This graph shows several best performing teams in terms of goals scored away differentials, Bilbao, Barcelona, Celta de Vigo, Real Madrid, Real Sociedad, Sevilla and Valencia had significant improvement over several seasons compared to their previous season. Teams like Sevilla had a bad season in terms of our metric in the 2015/2016 season.¶
Interest 3a: We are now interested in computing and plotting the total goals scored (home or away) on each season for each team. This way we can see if a team is performing very well in terms of total goals scored. We would also like to see if any team has performed better or worse over time. A simple bar chart with each team on the x-axis and total goals scored on the y-axis, aggregated over each season should help us get an idea.¶
Question 3a: How many goals has each team scored on each season?¶
Answer 3a: Total goals scored per Season¶
In [13]:
season_team_goal_sum = season_team_goal_sum.reset_index()
season_team_goal_sum.rename(columns={'Season': 'Season', 'Team_Home': 'Team', 0: 'Team_Goal'}, inplace=True)
season_team_goal_sum = reset_index_pivot_plot_series('stacked_bar', '', season_team_goal_sum, \
'Team', 'Season', 'Team_Goal', 'Total goals scored per Season', 'Team', 'Total Goals Scored', False, True)
season_team_goal_sum.head()
Out[13]:
Conclusion 3a: This graph shows two best performing team in terms of goals scored home, FC Barcelona and Real Madrid. Looking at the chart though is not that easy to notice any improvement or deterioration over time for each team. This is a limitation of this specific chart that we will explore next.¶
If we display the same information, but this time using a heat map, we can clearly see how each team has performed over time. For example we can see that both FC Barcelona and Real Madrid peaked their performance in the 2011/2012 season as well as in the 2014/2015 and 2015/2016 season.¶
In [14]:
season_team_goal_sum = reset_index_pivot_plot_series('heat_map', 'YlGnBu', season_team_goal_sum, \
'Team', 'Season', 'Team_Goal', 'Total goals scored per Season', 'Season', 'Team', False, False)
Interest 3b: Next, we are interested in computing and plotting which team performed better or worse over time in terms of total goals scored. A heatmap chart with each team on the y-axis and total goals scored differentials (goal difference from season to season) as values, with each season on the x-axis, should give us an idea.¶
Question 3b: How teams improved over time and what teams improved the most and the least in terms of total goals scored over time?¶
Answer 3b: Total goals scored over Seasons¶
In [15]:
season_team_goal_sum_diff = season_team_goal_sum.diff(periods=1, axis='columns')
season_team_goal_sum_diff = reset_index_pivot_plot_series('heat_map', 'cool', season_team_goal_sum_diff, \
'Team', 'Season', 'Team_Goal', 'Total goals scored differentials over Seasons', 'Season', 'Team', False, False)
Conclusion 3b: This graph shows several best performing teams in terms of total goals scored differentials, Bilbao, Barcelona, Real Madrid, Real Sociedad and Valencia. Real Sociedad had a stellar improvement in the 2012/2013 season compared to its previous season. Teams like Bilbao and Valencia had a bad season in terms of our metric in the 2014/2015 and 2015/2016 season correspondingly.¶
In [16]:
season_team_home_allowed_sum = spain_liga_bbva_df.groupby(['Season', 'Team_Home'])['Team_Away_Goal'].sum()
season_team_away_allowed_sum = spain_liga_bbva_df.groupby(['Season', 'Team_Away'])['Team_Home_Goal'].sum()
season_team_allowed_sum = season_team_home_allowed_sum + season_team_away_allowed_sum
print type(season_team_home_allowed_sum)
print season_team_home_allowed_sum.head(30)
Interest 4a: We are interested in computing and plotting the goals allowed home on each season for each team. This way we can see if a team is performing very bad in terms of goals allowed home. We would also like to see if any team has performed better or worse over time. A simple bar chart with each team on the x-axis and goals allowed home on the y-axis, aggregated over each season should help us get an idea.¶
Question 4a: How many goals has each team allowed home on each season?¶
Answer 4a: Home goals allowed per Season¶
In [17]:
season_team_home_allowed_sum = reset_index_pivot_plot_series('stacked_bar', '', season_team_home_allowed_sum, \
'Team_Home', 'Season', 'Team_Away_Goal', 'Home goals allowed per Season', 'Team', 'Home Goals Allowed', True, True)
season_team_home_allowed_sum.head()
Out[17]:
Conclusion 4a: This graph shows two worst performing team in terms of goals allowed home, Getafe CF and RCD Espanyol. Looking at the chart though is not that easy to notice any improvement or deterioration over time for each team. This is a limitation of this specific chart that we will explore next.¶
If we display the same information, but this time using a heat map, we can clearly see how each team has performed over time. For example we can see that Rayo Vallecano, Real Sporting de Gijon and Almeria had their worst performance in the 2013/2014, 2008/2009 and 2010/2011 season correspondingly.¶
In [18]:
season_team_home_allowed_sum = reset_index_pivot_plot_series('heat_map', 'RdPu', season_team_home_allowed_sum, \
'Team_Home', 'Season', 'Team_Away_Goal', 'Home goals allowed per Season', 'Season', 'Team', False, False)
Interest 4b: Next, we are interested in computing and plotting which team performed better or worse over time in terms of goals allowed home. A heatmap chart with each team on the y-axis and goals allowed home differentials (goal difference from season to season) as values, with each season on the x-axis, should give us an idea.¶
Question 4b: How teams deteriorated over time and what teams deteriorated the most and the least in terms of goals allowed home over time?¶
Answer 4b: Home goals allowed over Seasons¶
In [19]:
season_team_home_allowed_sum_diff = season_team_home_allowed_sum.diff(periods=1, axis='columns')
season_team_home_allowed_sum_diff = reset_index_pivot_plot_series('heat_map', 'magma_r', season_team_home_allowed_sum_diff, \
'Team_Home', 'Season', 'Team_Away_Goal', 'Home goals allowed differentials over Seasons', 'Season', 'Team', False, False)
Conclusion 4b: This graph shows three worst performing teams in terms of goals allowed home differentials, Elche CF in their 2014/2015 season, RCD Mallorca in their 2012/2013 season and Real Valladolid in their 2009/2010 season, compared to their previous seasons as always.¶
Interest 5a: Similarly to goals allowed home, we are also interested in computing and plotting the goals allowed away on each season for each team. This way we can see if a team is performing very bad in terms of goals allowed away. We would also like to see if any team has performed better or worse over time. A simple bar chart with each team on the x-axis and goals allowed away on the y-axis, aggregated over each season should help us get an idea.¶
Question 5a: How many goals has each team allowed away on each season?¶
Answer 5a: Away goals allowed per Season¶
In [20]:
season_team_away_allowed_sum = reset_index_pivot_plot_series('stacked_bar', '', season_team_away_allowed_sum, \
'Team_Away', 'Season', 'Team_Home_Goal', 'Away goals allowed per Season', 'Team', 'Away Goals Allowed', True, True)
season_team_away_allowed_sum.head()
Out[20]:
Conclusion 5a: This graph shows two worst performing teams in terms of goals allowed away, Getafe CF and RCD Espanyol. Looking at the chart though is not that easy to notice any improvement or deterioration over time for each team. This is a limitation of this specific chart that we will explore next.¶
If we display the same information, but this time using a heat map, we can clearly see how each team has performed over time. For example we can see that teams like CD Numancia, Getafe, Grenada, Rayo, Betis Balompie had some underperforming seasons in 2008, 2015, 2014 2011, 2013 correspondingly.¶
In [21]:
season_team_away_allowed_sum = reset_index_pivot_plot_series('heat_map', 'RdPu', season_team_away_allowed_sum, \
'Team_Away', 'Season', 'Team_Home_Goal', 'Away goals allowed per Season', 'Season', 'Team', False, False)
Interest 5b: Next, we are interested in computing and plotting which team performed better or worse over time in terms of goals allowed away. A heatmap chart with each team on the y-axis and goals allowed away differentials (goal difference from season to season) as values, with each season on the x-axis, should give us an idea.¶
Question 5b: How teams deteriorated over time and what teams deteriorated the most and the least in terms of goals allowed away over time?¶
Answer 5b: Away goals allowed over Seasons¶
In [22]:
season_team_away_allowed_sum_diff = season_team_away_allowed_sum.diff(periods=1, axis='columns')
season_team_away_allowed_sum_diff = reset_index_pivot_plot_series('heat_map', 'magma_r', season_team_away_allowed_sum_diff, \
'Team_Away', 'Season', 'Team_Home_Goal', 'Away goals allowed differentials over Seasons', 'Season', 'Team', False, False)
Conclusion 5b: This graph shows four worst performing teams in terms of goals allowed away differentials, Grenada in their 2014/2015 season, RCD Espanyol in their 2015/2016 season, Gijon in their 2011/2012 season and Eibar in their 2015/2016 season, compared to their previous seasons as always.¶
Interest 6a: We are now interested in computing and plotting the total goals allowed, home or away, on each season for each team. This way we can see if a team is performing very bad in terms of total goals allowed. We would also like to see if any team has performed better or worse over time. A simple bar chart with each team on the x-axis and total goals allowed on the y-axis, aggregated over each season should help us get an idea¶
Question 6a: How many goals has each team allowed on each season?¶
Answer 6a: Total goals allowed per Season¶
In [23]:
season_team_allowed_sum = season_team_allowed_sum.reset_index()
season_team_allowed_sum.rename(columns={'Season': 'Season', 'Team_Home': 'Team', 0: 'Team_Allowed'}, inplace=True)
season_team_allowed_sum = reset_index_pivot_plot_series('stacked_bar', '', season_team_allowed_sum, \
'Team', 'Season', 'Team_Allowed', 'Total goals allowed per Season', 'Team', 'Total Goals Allowed', False, True)
season_team_allowed_sum.head()
Out[23]:
Conclusion 6a: This graph shows two worst performing team in terms of total goals allowed, Getafe CF and RCD Espanyol. Looking at the chart though is not that easy to notice any improvement or deterioration over time for each team. This is a limitation of this specific chart that we will explore next.¶
If we display the same information, but this time using a heat map, we can clearly see how each team has performed over time. For example we can see that Rayo Vallecano, Real Betis and Gijon had their worst performance in the 2013/2014, 2013/2014 and 2008/2009 season correspondingly.¶
In [24]:
season_team_allowed_sum = reset_index_pivot_plot_series('heat_map', 'RdPu', season_team_allowed_sum, \
'Team', 'Season', 'Team_Allowed', 'Total goals allowed per Season', 'Season', 'Team', False, False)
Interest 6b: Next, we are interested in computing and plotting which team performed better or worse over time in terms of total goals allowed. A heatmap chart with each team on the y-axis and total goals allowed differentials (goal difference from season to season) as values, with each season on the x-axis, should give us an idea.¶
Question 6b: How teams deteriorated over time and what teams deteriorated the most and the least in terms of goals allowed over time?¶
Answer 6b: Total goals allowed over Seasons¶
In [25]:
season_team_allowed_sum_diff = season_team_allowed_sum.diff(periods=1, axis='columns')
season_team_allowed_sum = reset_index_pivot_plot_series('heat_map', 'magma_r', season_team_allowed_sum_diff, \
'Team', 'Season', 'Team_Allowed', 'Total goals allowed differentials over Seasons', 'Season', 'Team', False, False)
Conclusion 6b: This graph shows three worst performing teams in terms of goals allowed home differentials, Levante in their 2014/2015 season, Mallorca in their 2012/2013 season and Gijon in their 2011/2012 season, compared to their previous seasons as always.¶
Compute number of unique seasons¶
In [26]:
unique_seasons = pd.Series(spain_liga_bbva_df['Season'].unique())
A reminder of the structure of our original League DataFrame¶
In [27]:
spain_liga_bbva_df.head()
Out[27]:
Group original spain_liga_bbva_df DataFrame by Team_Home and Team_Away goals scored and allowed¶
In [28]:
team_match_goal_sum = spain_liga_bbva_df.groupby(['Team_Home', 'Team_Away'])['Team_Home_Goal', 'Team_Away_Goal'].sum()
team_match_goal_sum.head()
Out[28]:
Interest 7a: We are interested in computing and plotting goals scored home for each team against each other team. This way we can see if a team is performing very well in terms of goals scored home, over all seasons. A heatmap, with each team playing home on the y-axis and each team playing away on the x-axis and goals scored home as values, should help us get an idea.¶
Question 7a: How many goals has each team scored home, against other teams, over all seasons?¶
Answer 7a: Home goals scored against other teams¶
In [29]:
team_match_home_goal_sum = reset_index_pivot_plot_series('heat_map', 'YlGnBu', team_match_goal_sum, \
'Team_Home', 'Team_Away', 'Team_Home_Goal', \
'Home goals scored over '+str(unique_seasons.size)+' Seasons: '+unique_seasons.iloc[0]+' to '+unique_seasons.iloc[-1], \
'Team Away', 'Team Home', True, True)
team_match_home_goal_sum.head()
Out[29]:
Conclusion 7a: This graph shows two best performing team in terms of goals scored home, FC Barcelona and Real Madrid. It also shows few of the worst performing teams like Numancia, Cordoba and Deportivo. Interestingly, Bilbao and Getafe seem to not perform that well against the two top teams when they play at their home fields.¶
Interest 7b: We are interested in computing and plotting goals scored away for each team against each other team. This way we can see if a team is performing very well in terms of goals scored away, over all seasons. A heatmap, with each team playing away on the y-axis and each team playing home on the x-axis and goals scored away as values, should help us get an idea.¶
Question 7b: How many goals has each team scored away, against other teams, over all seasons?¶
Answer 7b: Away goals scored against other teams¶
In [30]:
team_match_away_goal_sum = reset_index_pivot_plot_series('heat_map', 'YlGnBu', team_match_goal_sum, \
'Team_Away', 'Team_Home', 'Team_Away_Goal', \
'Away goals scored over '+str(unique_seasons.size)+' Seasons: '+unique_seasons.iloc[0]+' to '+unique_seasons.iloc[-1], \
'Team Home', 'Team Away', True, True)
team_match_away_goal_sum.head()
Out[30]:
Conclusion 7b: This graph shows two best performing team in terms of goals scored home, FC Barcelona and Real Madrid. It also shows few of the worst performing teams like Numancia, Cordoba and Deportivo. Interestingly, Getafe and La Coruna seem to not perform that well against the two top teams when they play away from their home fields.¶
Interest 7c: We are interested in computing and plotting total goals scored, home or away, for each team against each other team. This way we can see if a team is performing very well in terms of total goals scored, over all seasons. A heatmap, with each team on the y-axis and each team on the x-axis and goals scored from each team on the y-axis over each team on the x-axis as values, should help us get an idea.¶
Question 7c: How many goals has each team scored, against other teams, over all seasons¶
Answer 7c: Total goals scored against other teams¶
In [31]:
team_match_total_goal_sum = team_match_home_goal_sum + team_match_away_goal_sum
team_match_total_goal_sum = reset_index_pivot_plot_series('heat_map', 'YlGnBu', team_match_total_goal_sum, \
'', '', '', \
'Total goals scored over '+str(unique_seasons.size)+' Seasons: '+unique_seasons.iloc[0]+' to '+unique_seasons.iloc[-1], \
'Team', 'Team', False, False)
team_match_total_goal_sum.head()
Out[31]:
Conclusion 7c: This graph shows two best performing team in terms of total goals scored, home or away, FC Barcelona and Real Madrid. It also shows few of the worst performing teams like Numancia, Cordoba and Deportivo. Interestingly, Bilbao and Getafe seem to not perform that well against the two top teams, regardless if they play at their home fields or away.¶
Interest 8a: We are interested in computing and plotting goals allowed home for each team against each other team. This way we can see if a team is performing very bad in terms of goals allowed home, over all seasons. A heatmap, with each team playing home on the y-axis and each team playing away on the x-axis and goals allowed home as values, should help us get an idea.¶
Question 8a: How many goals has each team allowed home, from other teams, over all seasons?¶
Answer 8a: Home goals allowed from other teams¶
In [32]:
team_match_home_allowed_sum = reset_index_pivot_plot_series('heat_map', 'RdPu', team_match_goal_sum, \
'Team_Home', 'Team_Away', 'Team_Away_Goal', \
'Home goals allowed over '+str(unique_seasons.size)+' Seasons: '+unique_seasons.iloc[0]+' to '+unique_seasons.iloc[-1], \
'Team Away', 'Team Home', True, True)
team_match_home_allowed_sum.head()
Out[32]:
Conclusion 8a: This graph shows teams like Getafe, Levante and La Coruna seem to not perform that well against other teams and especially against the two top teams, Barcelona and Real Madrid.¶
Interest 8b: We are interested in computing and plotting goals allowed away for each team against each other team. This way we can see if a team is performing very bad in terms of goals allowed away, over all seasons. A heatmap, with each team playing away on the y-axis and each team playing home on the x-axis and goals allowed away as values, should help us get an idea.¶
Question 8b: How many goals has each team allowed away, from other teams, over all seasons?¶
Answer 8b: Away goals allowed from other teams¶
In [33]:
team_match_away_allowed_sum = reset_index_pivot_plot_series('heat_map', 'RdPu', team_match_goal_sum, \
'Team_Away', 'Team_Home', 'Team_Home_Goal', \
'Away goals allowed over '+str(unique_seasons.size)+' Seasons: '+unique_seasons.iloc[0]+' to '+unique_seasons.iloc[-1], \
'Team Home', 'Team Away', True, True)
team_match_away_allowed_sum.head()
Out[33]:
Conclusion 8b: This graph shows teams like Bilbao, Atletico Madrid and Getafe seem to not perform that well against other teams and especially against the two top teams, Barcelona and Real Madrid.¶
Interest 8c: We are interested in computing and plotting total goals allowed, home or away, for each team against each other team. This way we can see if a team is performing very bad in terms of total goals allowed, over all seasons. A heatmap, with each team on the y-axis and each team on the x-axis and goals allowed from each team on the y-axis from each team on the x-axis as values, should help us get an idea.¶
Question 8c: How many goals has each team allowed, from other teams, over all seasons¶
Answer 8c: Total goals allowed from other teams¶
In [34]:
team_match_total_allowed_sum = team_match_home_allowed_sum + team_match_away_allowed_sum
team_match_total_allowed_sum = reset_index_pivot_plot_series('heat_map', 'RdPu', team_match_total_allowed_sum, \
'', '', '', \
'Total goals allowed over '+str(unique_seasons.size)+' Seasons: '+unique_seasons.iloc[0]+' to '+unique_seasons.iloc[-1], \
'Team', 'Team', False, False)
team_match_total_allowed_sum.head()
Out[34]:
Conclusion 8c: This graph shows teams like Bilbao, Atletico Madrid and Getafe seem to not perform that well against other teams and especially against the two top teams, Barcelona and Real Madrid.¶
Apart from individual goals scored or allowed, let us consider a match outcome in terms of win (more goals scored), tie (no goals or equal goals scored) or loss (rival team scored more goals) for each team.¶
In [35]:
"""
categorize_win_tie_loss(team_match_df) is used for categorizing the outcome of match as a win, tie or loss for the home team.
Args:
team_match_df: Team match DataFrame with columns 'Team_Home_Goal' and 'Team_Away_Goal' available.
Returns:
Updated DataFrame with 3 new columns created in case of a match home win, tie or loss.
Raises:
None
"""
def categorize_win_tie_loss(team_match_df):
#Reference
#https://pandas.pydata.org/pandas-docs/stable/indexing.html#setting-with-enlargement
team_match_df.loc[:, 'Team_Home_Win'] = 0
team_match_df.loc[:, 'Team_Home_Tie'] = 0
team_match_df.loc[:, 'Team_Away_Win'] = 0
team_match_df.loc[team_match_df['Team_Home_Goal'] > team_match_df['Team_Away_Goal'], 'Team_Home_Win'] = 1
team_match_df.loc[team_match_df['Team_Home_Goal'] == team_match_df['Team_Away_Goal'], 'Team_Home_Tie'] = 1
team_match_df.loc[team_match_df['Team_Home_Goal'] < team_match_df['Team_Away_Goal'], 'Team_Away_Win'] = 1
return team_match_df
Enhance our original League Dataframe with win, tie or loss outcome for each match.¶
In [36]:
spain_liga_bbva_df = categorize_win_tie_loss(spain_liga_bbva_df)
spain_liga_bbva_df.head()
Out[36]:
Group original League DataFrame by Team_Home and Team_Away win, tie or loss after each match¶
In [37]:
team_match_outcome_sum = spain_liga_bbva_df.groupby(['Team_Home', 'Team_Away']) \
['Team_Home_Win', 'Team_Home_Tie', 'Team_Away_Win'].sum()
team_match_outcome_sum.head()
Out[37]:
Let's set a new index for our team_match_outcome_sum Dataframe as the pair of home and away teams of a match¶
https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.set_index.html¶
In [38]:
team_match_outcome_sum = team_match_outcome_sum.reset_index()
team_match_outcome_sum = team_match_outcome_sum.set_index(['Team_Home', 'Team_Away'])
Create a function that for each pair of teams (team1, team2) will add home wins for team1 with away wins for team1 for each pair of teams (team2, team1). Similarly for ties.¶
In [39]:
"""
add_home_and_away_wins_ties(team_match_df) is used to keep track of wins earned home or away for each pair of teams.
Similarly for ties, home or away.
Args:
team_match_df: Team match DataFrame with columns ['Team_Home', 'Team_Away', 'Team_Home_Win', 'Team_Home_Tie', 'Team_Away_Win'] available.
Returns:
Updated DataFrame with 2 new columns created, one for total wins, one for total ties.
Raises:
None
"""
def add_home_and_away_wins_ties(team_match_df):
team_match_df.loc[:, 'Team_Win'] = 0
team_match_df.loc[:, 'Team_Tie'] = 0
for pair in team_match_outcome_sum.index:
team_match_df.loc[(pair[0], pair[1]), :]['Team_Win'] = team_match_df.loc[(pair[0], pair[1]), :]['Team_Home_Win'] + team_match_df.loc[(pair[1], pair[0]), :]['Team_Away_Win']
team_match_df.loc[(pair[0], pair[1]), :]['Team_Tie'] = team_match_df.loc[(pair[0], pair[1]), :]['Team_Home_Tie'] + team_match_df.loc[(pair[1], pair[0]), :]['Team_Home_Tie']
return team_match_df
In [40]:
team_match_outcome_sum = add_home_and_away_wins_ties(team_match_outcome_sum)
In [41]:
team_match_outcome_sum.head()
Out[41]:
Interest 9a: We are interested in computing and plotting number of wins for each team against each other team. This way we can see if a team is performing very well in terms of total wins, over all seasons. A heatmap, with each team on the y-axis and each team on the x-axis and total wins from each team on the y-axis over each team on the x-axis as values, should help us get an idea.¶
Question 9a: How many times did each team win, against other teams, over all seasons?¶
Answer 9a: Wins against other teams¶
In [42]:
team_match_win_sum = reset_index_pivot_plot_series('heat_map', 'autumn_r', team_match_outcome_sum, \
'Team_Home', 'Team_Away', 'Team_Win', \
'Wins over '+str(unique_seasons.size)+' Seasons: '+unique_seasons.iloc[0]+' to '+unique_seasons.iloc[-1], \
'Team', 'Team', True, True)
team_match_win_sum.head()
Out[42]:
Conclusion 9a: This graph shows three best performing teams in terms of total wins against other teams, home or away, FC Barcelona, Real Madrid and Atletico Madrid. It also shows few of the worst performing teams like Numancia, Cordoba and Deportivo.¶
Interest 9b: We are interested in computing and plotting number of ties for each team against each other team. This way we can see if a team is performing equally compared to other team, in terms of total ties, over all seasons. A heatmap, with each team on the y-axis and each team on the x-axis and total ties for each team on the y-axis against each team on the x-axis as values, should help us get an idea.¶
Question 9b: How many times did each team tie, against other teams, over all seasons?¶
Answer 9b: Ties against other teams¶
Note that this is the first time we encounter a symmetric matrix where entry[team1, team2] is the same as entry[team2, team1]¶
In [43]:
team_match_tie_sum = reset_index_pivot_plot_series('heat_map', 'rainbow', team_match_outcome_sum, \
'Team_Home', 'Team_Away', 'Team_Tie', \
'Home ties over '+str(unique_seasons.size)+' Seasons: '+unique_seasons.iloc[0]+' to '+unique_seasons.iloc[-1], \
'Team', 'Team', True, True)
team_match_tie_sum.head()
Out[43]:
Conclusion 9b: This graph shows that 2 pairs of teams seem to perform equally well, when each team within each pair, plays against the other team, Malaga vs Bilbao for a total of 8 ties and Espanyol vs Villarreal for a total of 7 ties.¶
In [44]:
spain_liga_bbva_stats = spain_liga_bbva_df.describe()
spain_liga_bbva_stats['Home_Advantage'] = spain_liga_bbva_stats['Team_Home_Goal'] / spain_liga_bbva_stats['Team_Away_Goal']
In [45]:
spain_liga_bbva_stats
Out[45]:
Looking at the Home_Advantage column above we see that on average, a team scores 1.4 more goals when playing home compared to goals scored away. So there is a home advantage after all!¶
Possible future work: We explored several questions and we saw first hand the power and limitation of each visualization we used.¶
In the near future we could drill down into each team, especially the top 2 performing teams, and see what player attributes contributed the most in their wins or which players had the most penalties or what team attributes lead to the most victories?¶
Try it on your own! Feel free to take a deep dive into another Country and League¶
Simply go to cell 8 above where we defined our query to our local sqlite database and change the Country and League name filters to your own favorites.¶
...
Country = 'Spain' AND
League.name = 'Spain LIGA BBVA'
...
Everything until this point will work regardless of which Country and League you selected.¶
The remaining short part applies only to the Spanish BBVA League. As a reminder, you can adopt it to your own rival teams within your favorite League.¶
El Clásico¶
El Clásico is the name given in football to any match between fierce rivals FC Barcelona and Real Madrid.¶
https://en.wikipedia.org/wiki/El_Clásico¶
All El Clásico matches in our dataset¶
In [46]:
FCB = 'FC Barcelona'
RMCF = 'Real Madrid CF'
spain_liga_bbva_df[((spain_liga_bbva_df['Team_Home'] == FCB) & (spain_liga_bbva_df['Team_Away'] == RMCF)) \
| ((spain_liga_bbva_df['Team_Home'] == RMCF) & (spain_liga_bbva_df['Team_Away'] == FCB))]
Out[46]:
Match outcomes in goals when 'FC Barcelona' played at home (see corresponding graph and questions 7a, 8a above)¶
In [47]:
spain_liga_bbva_df[((spain_liga_bbva_df['Team_Home'] == FCB) & (spain_liga_bbva_df['Team_Away'] == RMCF)) ][['Team_Home_Goal','Team_Away_Goal']].sum()
Out[47]:
Match outcomes in goals when 'Real Madrid CF' played at home (see corresponding graph and questions 7b, 8b above)¶
In [48]:
spain_liga_bbva_df[((spain_liga_bbva_df['Team_Home'] == RMCF) & (spain_liga_bbva_df['Team_Away'] == FCB)) ][['Team_Home_Goal','Team_Away_Goal']].sum()
Out[48]:
Match outcomes in number of wins of 'FC Barcelona' over 'Real Madrid CF' (see corresponding graph and questions 9a above)¶
In [49]:
team_match_win_sum.loc[(FCB, RMCF)]
Out[49]:
Match outcomes in number of wins of 'Real Madrid CF' over 'FC Barcelona' (see corresponding graph and questions 9a above)¶
In [50]:
team_match_win_sum.loc[(RMCF, FCB)]
Out[50]:
Match outcomes in number of ties of the pair 'FC Barcelona', 'Real Madrid CF' (see corresponding graph and questions 9b above)¶
In [51]:
team_match_tie_sum.loc[(FCB, RMCF)] #same as team_match_tie_sum.loc[(RMCF, FCB)] since matrix team_match_tie_sum is symmetric
Out[51]:
Additional Material - Appendix¶
Matplotlib colormap reference¶
https://matplotlib.org/examples/color/colormaps_reference.html
Personal favorites for displaying positive impact: 'YlGnBu', 'cool'¶
Personal favorites for displaying negative impact: 'RdPu', 'magma_r', 'plasma_r'¶
In [52]:
#plot_colors = ['Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', 'BuGn', 'BuGn_r', 'BuPu', 'BuPu_r', 'CMRmap', 'CMRmap_r', 'Dark2', 'Dark2_r', 'GnBu', 'GnBu_r', 'Greens', 'Greens_r', 'Greys', 'Greys_r', 'OrRd', 'OrRd_r', 'Oranges', 'Oranges_r', 'PRGn', 'PRGn_r', 'Paired', 'Paired_r', 'Pastel1', 'Pastel1_r', 'Pastel2', 'Pastel2_r', 'PiYG', 'PiYG_r', 'PuBu', 'PuBuGn', 'PuBuGn_r', 'PuBu_r', 'PuOr', 'PuOr_r', 'PuRd', 'PuRd_r', 'Purples', 'Purples_r', 'RdBu', 'RdBu_r', 'RdGy', 'RdGy_r', 'RdPu', 'RdPu_r', 'RdYlBu', 'RdYlBu_r', 'RdYlGn', 'RdYlGn_r', 'Reds', 'Reds_r', 'Set1', 'Set1_r', 'Set2', 'Set2_r', 'Set3', 'Set3_r', 'Spectral', 'Spectral_r', 'Vega10', 'Vega10_r', 'Vega20', 'Vega20_r', 'Vega20b', 'Vega20b_r', 'Vega20c', 'Vega20c_r', 'Wistia', 'Wistia_r', 'YlGn', 'YlGnBu', 'YlGnBu_r', 'YlGn_r', 'YlOrBr', 'YlOrBr_r', 'YlOrRd', 'YlOrRd_r', 'afmhot', 'afmhot_r', 'autumn', 'autumn_r', 'binary', 'binary_r', 'bone', 'bone_r', 'brg', 'brg_r', 'bwr', 'bwr_r', 'cool', 'cool_r', 'coolwarm', 'coolwarm_r', 'copper', 'copper_r', 'cubehelix', 'cubehelix_r', 'flag', 'flag_r', 'gist_earth', 'gist_earth_r', 'gist_gray', 'gist_gray_r', 'gist_heat', 'gist_heat_r', 'gist_ncar', 'gist_ncar_r', 'gist_rainbow', 'gist_rainbow_r', 'gist_stern', 'gist_stern_r', 'gist_yarg', 'gist_yarg_r', 'gnuplot', 'gnuplot2', 'gnuplot2_r', 'gnuplot_r', 'gray', 'gray_r', 'hot', 'hot_r', 'hsv', 'hsv_r', 'icefire', 'icefire_r', 'inferno', 'inferno_r', 'jet', 'jet_r', 'magma', 'magma_r', 'mako', 'mako_r', 'nipy_spectral', 'nipy_spectral_r', 'ocean', 'ocean_r', 'pink', 'pink_r', 'plasma', 'plasma_r', 'prism', 'prism_r', 'rainbow', 'rainbow_r', 'rocket', 'rocket_r', 'seismic', 'seismic_r', 'spectral', 'spectral_r', 'spring', 'spring_r', 'summer', 'summer_r', 'tab10', 'tab10_r', 'tab20', 'tab20_r', 'tab20b', 'tab20b_r', 'tab20c', 'tab20c_r', 'terrain', 'terrain_r', 'viridis', 'viridis_r', 'vlag', 'vlag_r', 'winter', 'winter_r']
plot_colors = ['YlGnBu', 'cool', 'RdPu', 'magma_r', 'plasma_r']
for plot_color in plot_colors:
season_team_home_goal_sum = reset_index_pivot_plot_series('heat_map', plot_color, season_team_home_goal_sum, \
'Team_Home', 'Season', 'Team_Home_Goal', plot_color + ' - Home goals scored per Season', 'Team', 'Home Goals Scored', False, False)
